In head mounted display (HMD) and other imaging systems, depth imaging may be obtained by correlating left and right stereoscopic images to match pixels between the stereoscopic images. The pixels may be matched by determining which pixels are the most similar between the left and right images. Pixels correlated between the left and right stereoscopic images may be used to determine depth information. For example, a disparity between the location of the pixel in the left image and the location of the corresponding pixel in the right image may be used to calculate the depth information using binocular disparity techniques. An image may be produced that contains depth information for a scene, such as information related to how deep or how far away objects in the scene are in relation to a camera's viewpoint. Such images are useful in perceptual computing for applications such as gesture tracking and object recognition, for example.